Skip to content

Enable more clippy lints - #24322

Merged
Dandandan merged 45 commits into
apache:mainfrom
emilk:emilk/more-clippy-lints
Aug 18, 2026
Merged

Enable more clippy lints#24322
Dandandan merged 45 commits into
apache:mainfrom
emilk:emilk/more-clippy-lints

Conversation

@emilk

@emilk emilk commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

More allow-by-default clippy lints that simplify code, catch bugs, or improve performance.
Every lint here was verified to be outside the default-warn groups, so none is a no-op.

What changes are included in this PR?

One commit per lint, including its fixes, so any single lint can be reverted on its own.
The first commit is the exception: 19 lints that had zero hits and need no code changes.

Two real bugs fell out of literal_string_with_formatting_args, where a {placeholder} was
printed verbatim instead of interpolated:

  • benchmarks/src/nlj.rs: "NLJ benchmark Q{query_id} failed…".to_string()
  • parquet_advanced_index.rs: .expect("metadata for file not found: {filename}")

fallible_impl_from also flagged that From<protobuf::Constraint> for Constraint panics on a
message with an unset constraint_mode. Fixing that needs a breaking change to TryFrom, so it
is only marked with #[expect] here.

Are these changes tested?

Covered by existing tests plus the clippy CI job. I also ran the extended test suite locally.

Are there any user-facing changes?

One non-breaking signature change: format_human_display and a few private helpers now take T
instead of Option<T> (clippy::single_option_map). No public API changes.

@github-actions github-actions Bot added sql SQL Planner logical-expr Logical plan and expressions physical-expr Changes to the physical-expr crates optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate common Related to common crate execution Related to the execution crate proto Related to proto crate functions Changes to functions implementation datasource Changes to the datasource crate ffi Changes to the ffi crate physical-plan Changes to the physical-plan crate spark labels Aug 13, 2026
Comment thread datafusion/core/tests/user_defined/user_defined_scalar_functions.rs Outdated
Comment thread datafusion/pruning/src/pruning_predicate.rs Outdated
Comment thread datafusion/spark/src/function/string/length.rs Outdated
Comment thread datafusion/expr/src/predicate_bounds.rs Outdated
@emilk emilk changed the title Enable 33 more clippy lints Enable more clippy lints Aug 13, 2026
Comment thread benchmarks/src/nlj.rs
Comment thread datafusion/core/tests/memory_limit/memory_limit_validation/sort_mem_validation.rs Outdated
Comment thread datafusion/datasource-parquet/src/opener/mod.rs
Comment thread datafusion/core/src/execution/context/mod.rs Outdated
Comment thread datafusion/common/src/scalar/mod.rs Outdated
emilk and others added 11 commits August 13, 2026 14:12
All of these are `allow` by default and currently have zero hits across
the workspace (`--all-targets --all-features`), so they act purely as
guards against future regressions:

* Bug catchers: `same_functions_in_if_condition`,
  `self_only_used_in_recursion`, `unchecked_time_subtraction`,
  `expl_impl_clone_on_copy`, `into_iter_without_iter`,
  `iter_without_into_iter`, `unnecessary_safety_doc`
* Performance: `large_stack_arrays`, `large_stack_frames`, `linkedlist`,
  `set_contains_or_insert`, `string_lit_chars_any`
* Simplification / API hygiene: `empty_enums`,
  `fn_params_excessive_bools`, `iter_not_returning_iterator`,
  `non_std_lazy_statics`, `ptr_cast_constness`, `pub_without_shorthand`,
  `trait_duplication_in_bounds`

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `.peekable()` in the sqllogictest Postgres engine was never peeked,
so it only added an extra layer of indirection.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Catches `From` implementations that can panic, where `TryFrom` would be
the honest signature.

All three existing hits would need a breaking API change to fix, so they
get `#[expect]` for now. Two of them (`Constraint`) panic on a protobuf
message with an unset `constraint_mode`, i.e. on malformed input.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Catches exact float comparisons against constants, e.g. `x == 0.0`.

The two existing hits in `value_transition!` really do want an exact
comparison against `f32::MIN`/`MAX`, so they get `#[expect]`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Catches float literals that silently round, e.g. `let x: f32 = 0.1234567890123;`.

The three existing hits are false positives: they spell out exact powers
of two (2^64 and 2^64-2^41), which float `Display` renders with fewer
digits, so the lint thinks precision was lost.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`(a + b) / 2` overflows when `a + b` exceeds the type's range;
`a.midpoint(b)` does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Catches strings that contain `{...}` but are never actually formatted.

This found two real bugs where the placeholder was silently printed
verbatim:
* `benchmarks/src/nlj.rs`: `"NLJ benchmark Q{query_id} failed…".to_string()`
* `parquet_advanced_index.rs`: `.expect("metadata for file not found: {filename}")`

The remaining hits are intentional: shell-style `${VAR:-default}`
placeholders, `{rows}`-style templates substituted with `str::replace`,
and braces inside expected struct output. Those get `#[expect]`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`&x as *const T` silently picks a pointer type; `std::ptr::from_ref(&x)`
keeps the referent type explicit and cannot accidentally change it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Catches `// SAFETY:` comments that do not sit in front of anything
unsafe, so that a `SAFETY:` comment reliably means "an unsafe block
follows, and here is why it is sound".

* Three comments documented an `unwrap` or a safe copy rather than
  unsafe code, so they lose the `SAFETY:` prefix.
* Three sat in front of an `if` while the `unsafe` block was inside it,
  so they move next to the block they justify.
* Two were prose false positives, where the lint matched "safety:" in
  the middle of a doc comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Catches large types passed by value, which forces a memcpy at every call.

The single existing hit is `HyperLogLog::new_with_registers`, whose 16 KiB
array is moved into the returned struct, so a reference would only add a
copy. It gets `#[expect]`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Catches `-> Box<T>` where the caller gains nothing from the indirection.

The single existing hit is a test helper that both takes and returns
`Box<Expr>` so it can hand back the same allocation, so it gets
`#[expect]`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
emilk and others added 10 commits August 13, 2026 14:12
`if !cond { panic!(msg) }` is `assert!(cond, msg)`, which states the
invariant instead of its negation.

One of clippy's rewrites produced a double negative
(`!...is_none()`); that one is written as `.is_some()` instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A `..` in a pattern that already binds every field does nothing today,
but silently swallows any field added later. Removing it turns that into
a compile error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`if let Some(true) = x` reads as a binding but is really an equality
check; `x == Some(true)` (or `matches!`) says so.

Clippy suggested one tuple comparison, `(is_valid, is_included) == (true,
Some(true))`; that one is written as a plain `&&` instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`x.deref()` and `x.deref_mut()` are the operator spelled the long way;
`&*x` / `&mut *x` is the idiomatic form and does not need `Deref` in
scope.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Addresses review feedback on the `equatable_if_let` commit: clippy
suggested `matches!` in a few places where a plain equality check reads
better.

`predicate_bounds.rs` uses `.ok() == Some(false)` because
`DataFusionError` does not implement `PartialEq`, so `== Ok(false)` does
not compile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* `test_parse_duration_with_overflow_check` uses `Duration::from_mins`
  for the `"…m"` input again, so the constructor mirrors the unit suffix
  in the string being parsed. That trips
  `duration_suboptimal_units`, so the test gets an `#[expect]` saying why.
* Restore the `TODO`/`Issue` comments above the ignored
  `sort_with_mem_limit_2_cols_2` test, keeping a short `#[ignore]` reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.62055% with 212 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.25%. Comparing base (bb45fb8) to head (fbd3571).
⚠️ Report is 22 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/proto-models/src/generated/pbjson.rs 0.00% 57 Missing ⚠️
...gregates/group_values/multi_group_by/dictionary.rs 94.98% 13 Missing and 16 partials ⚠️
...hysical-expr/src/expressions/similar_to_pattern.rs 91.56% 10 Missing and 10 partials ⚠️
datafusion/common/src/utils/memory.rs 93.68% 17 Missing and 2 partials ⚠️
datafusion/spark/src/function/math/modulus.rs 88.40% 6 Missing and 10 partials ⚠️
...ysical-plan/src/aggregates/group_values/metrics.rs 88.54% 1 Missing and 14 partials ⚠️
...lan/src/joins/piecewise_merge_join/classic_join.rs 74.19% 0 Missing and 8 partials ⚠️
datafusion/physical-plan/src/union.rs 77.27% 0 Missing and 5 partials ⚠️
...ion/datasource-parquet/src/projection_read_plan.rs 95.78% 4 Missing ⚠️
datafusion/optimizer/src/decorrelate.rs 80.00% 1 Missing and 3 partials ⚠️
... and 19 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24322      +/-   ##
==========================================
+ Coverage   81.18%   81.25%   +0.07%     
==========================================
  Files        1110     1112       +2     
  Lines      388915   391509    +2594     
  Branches   388915   391509    +2594     
==========================================
+ Hits       315729   318116    +2387     
- Misses      54598    54681      +83     
- Partials    18588    18712     +124     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

`ab12f5e4b` ("fix(ffi): preserve TableProvider DML overrides") landed on
main after this branch was measured and added a new
`[x].into_iter()`, which the `iter_on_single_items` lint enabled here
rejects. CI builds the merge commit, so it failed there but not locally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@emilk
emilk force-pushed the emilk/more-clippy-lints branch from 10b33a9 to 1755ad1 Compare August 13, 2026 12:16
@emilk
emilk marked this pull request as ready for review August 13, 2026 12:21
@Dandandan

Copy link
Copy Markdown
Contributor

@emilk thanks, looks much better

@Dandandan
Dandandan added this pull request to the merge queue Aug 15, 2026
@Dandandan

Copy link
Copy Markdown
Contributor

TY @emilk

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 15, 2026
@Dandandan

Copy link
Copy Markdown
Contributor

@emilk there seems to be a new CI failure

emilk and others added 3 commits August 18, 2026 05:48
…ints

# Conflicts:
#	datafusion/optimizer/src/decorrelate.rs
The upstream merge brought in code written before these lints were
enabled, so it failed `clippy -D warnings` on the merge commit:

- `branches_sharing_code`: hoist `raw_keys` out of the if/else in
  `dictionary.rs`
- `iter_on_single_items`: `[None].iter()` -> `std::iter::once(&None)`
- `elidable_lifetime_names`: elide `'a` in `scan_with_args_inner`,
  `infer_boxed`, and `infer_options_boxed`

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the catalog Related to the catalog crate label Aug 18, 2026
@emilk

emilk commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

should be good now, hopefully

@Dandandan
Dandandan added this pull request to the merge queue Aug 18, 2026
Merged via the queue into apache:main with commit 303ff78 Aug 18, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

catalog Related to the catalog crate common Related to common crate core Core DataFusion crate datasource Changes to the datasource crate execution Related to the execution crate ffi Changes to the ffi crate functions Changes to functions implementation logical-expr Logical plan and expressions optimizer Optimizer rules physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate proto Related to proto crate spark sql SQL Planner sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants